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A Mixed Methods Approach to Determining the Impact of a Geoscience 
Field Research Program Upon Science Teachers’ Knowledge, Beliefs, 
and Instructional Practices 
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ABSTRACT 

A mixed methods research approach was used to investigate the impact of a geosciences research institute upon 62 science 
teachers' knowledge, beliefs, and teaching practices related to teaching the geosciences. Pre- and postinstitute quantitative 
and qualitative assessments revealed mixed results. Results of a quantitative measure found a statistically significant increase 
in the teachers' self-reported overall knowledge of the geosciences, knowledge of watersheds, and knowledge of brownfield 
sites. However, a more specific quantitative instrument that measured watershed knowledge indicated no change except in 
the teachers' awareness that the water quality in different towns is related. In addition, their science teaching self-efficacy, 
which was used as a quantitative measure of the teachers' beliefs about teaching science in general, showed a statistically 
significant decrease. Focus group discussions provided an indication as to why the teachers' self-efficacy decreased, as 
teachers described numerous barriers that limited their capacity to engage their students in field-based research projects even 
while noting the benefits of such an experience. Although the study was limited in the ability to measure direct impact on 
instructional practices, the teachers' perception of their confidence in their capability to use inquiry-based instruction in the 
classroom showed a statistically significant increase. The discrepant results provide useful information about the complex 
impact of a professional development experience upon teachers, while identifying possible limitations of the experience and of 
the assessment measures, as well as providing recommendations for future research. © 2016 National Association of Geoscience 
Teachers. [DOI: 10.5408/14-064.1] 
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INTRODUCTION 

The lackluster performance of U.S. students on state, 
national, and international science assessments has prompt¬ 
ed movement towards reforming the science curriculum and 
pedagogy in many public schools (Duncan, 2010; NGSS 
Lead States, 2013a). Among the goals of national science 
education reform is to address the needs of a national 
student audience while simultaneously making the content 
relevant to a specific context (Hollweg and Hill, 2003; NGSS 
Lead States, 2013b). One way to do this is by linking the 
content to a particular community or local issue through a 
pedagogy called place-based education (Dentzau, 2014; 
Hackworth, 2015). Placed-based education is commonly 
defined as the process of using the local cultural, historical, 
and physical environment to teach subjects across the 
curriculum in a way that emphasizes one's connection and 
responsibility to the community (Woodhouse and Knapp, 
2000; Smith, 2002; Sobel, 2004; Semkin, 2005). It also 
includes the emotional connection one has to a place 
(Williams and Semken, 2011). Although this pedagogy has 
its roots in environmental education, it is currently being 
implemented in a wide variety of settings, including 
geoscience education (e.g., LaSage et al., 2006; Semkin, 
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2005; Monet and Greene, 2012; DeFelice et al., 2014; 
Hougham et al., 2015). 

Given place-based education's focus on the develop¬ 
ment of the whole individual in relation to their environ¬ 
ment, it is not surprising that this pedagogy has been 
demonstrated to have a positive impact on students and 
teachers. Place-based education has been shown to increase 
students' social responsibility and knowledge of the local 
community while building their content knowledge in a way 
that is relevant and engaging (State Education and 
Environment Roundtable, 2000; Athman and Monroe, 
2004; Jennings et al., 2005). Teachers who use a local 
context for their instruction experience increased satisfaction 
in teaching, in part because there are fewer student 
discipline problems (Meichtry and Smith, 2007). 

Local environments and their inhabitants are complex 
and dynamic settings that often have their own distinct 
characteristics. In order to capture the impact of an 
educational experience that implements place-based peda¬ 
gogy, it is important to employ a correspondingly complex 
research methodology such as the mixed methods approach. 
This research methodology is well suited to capture both the 
cognitive and affective impact of a place-based geoscience 
program, since a wide variety of quantitative as well as 
qualitative measures can be used to illuminate program 
impact in a way that one type of measure alone is not able to 
do Qohnson and Turner, 2003; Ponce and Pagan-Maldona- 
do, 2015). Quantitative measures are designed to investigate 
researcher-defined constructs (Patton, 1990), such as knowl¬ 
edge of the water cycle. In addition, they can even measure 
likelihood to engage in a future behavior, such as proper 
disposal of hazardous wastes. In contrast, qualitative data 
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FIGURE 1: Map of Rouge watershed. 
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allow participants to describe their own experience and how 
it fits into their unique context. Mixed methods research has 
the ability to capture the complicated and potentially 
contradictory impacts of educational programs (Phillips, 
2009). 

Program Description 

In 2006, the University of Michigan-Dearborn (UMD) 
created a Geosciences Institute for Research and Education 
(GIRE). The GIRE is a consortium of faculty, UMD graduate 
and undergraduate students, and local geologists who have 
worked with local Earth Science teachers and their students 
in a 3 week, field-based program each summer for 7 y. The 
funding for the summer institutes came from two grants 
from the National Science Foundation's Opportunities for 
Enhancing Diversity in the Geosciences (OEDG) Program. 
The basis of the program was a hands-on research 
experience in the geosciences that demonstrated how 


knowledge of geology can be used to solve community- 
based environmental problems. 

A field research experience was included because these 
have been demonstrated to increase both teacher and 
student content knowledge and interest in the sciences 
(Barab and Hay, 2001; Buck, 2003; Pop et al., 2010), 
especially if placed in a local context such as a local river 
or watershed (O'Connell et al., 2004; Meichtry and Smith, 
2007). Since the university personnel, teachers, and their 
students all lived within the same Rouge watershed (Fig. 1), 
the watershed was used as the local context for the program. 

The overall goals of the GIRE were: 

• to provide an awareness of opportunities in the 
geosciences to groups of underrepresented minority 
students; 

• to improve teacher knowledge of geology; 

• to stimulate student interest in science; and 
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• to increase enrollment and retention of groups of 
underrepresented minority students in the geoscienc¬ 
es at UMD. 

Each summer, the GIRE workshops consisted of three 5 d 
sessions. Each session consisted of 1.5 d of interactive 
workshops where participants learned how to use field/ 
laboratory instrumentation and went on field trips to gain a 
local perspective, and 3.5 d of hands-on field and laboratory 
activities. Session leaders, who were often working profes¬ 
sionals, emphasized during the field trips and activities how 
the geosciences are uniquely positioned to help solve many 
local environmental problems. The goal was to not have the 
teachers and their students complete research projects, but 
rather to involve them in active, ongoing research taking place 
in their community. This was achieved, as participants were 
introduced to the different field and laboratory activities 
necessary to solve the problems. 

The field research projects focused on watershed issues, 
since the watershed, or catchment basin, is an excellent unit 
for the integration of geoscience research, and it is consistent 
with a geoscience theme, i.e., solving environmental 
problems (Murray et al., 2004; Kaufman et al., 2011). The 
watershed approach is appropriate because it focuses on 
efforts to address solutions to environmental problems 
within hydrologically defined geographic areas (U.S. Envi¬ 
ronmental Protection Agency, 1996). During the GIRE, the 
teachers and students conducted research within the 
geological features of the Rouge watershed as they learned 
about the history of the region and the environmental issues 
located within the watershed. 

The geoscience research projects at UMD in which the 
teachers and students participated included an investigation 
of soil-lead contamination in Delray, an impoverished 
community in southwest Detroit that has a history of 
pollution by industries (Graham, 2015). The teachers and 
student participants collected soil samples from residential 
properties throughout the Delray community following 
Environmental Protection Agency (EPA) protocol. 

In a second project, the participants investigated the 
impact of land use on groundwater and surface-water 
quality by installing groundwater monitoring wells along 
the lower branch of the Rouge River. The students and 
teachers then collected groundwater, sediment, and surface- 
water samples for subsequent chemical analysis. This project 
indicated that the use of road salt as a de-icer in winter is 
having a profound effect on the chemistry and ecology of the 
Rouge River. 

A third project early in the GIRE focused on the 
transport and fallout of heavy metals from a municipal 
incinerator. These field research activities provided oppor¬ 
tunities for the teachers to deepen their knowledge of the 
geosciences as they performed research tasks that were 
focused on a local environmental problem. 

This study focuses on investigating the impact of the 
GIRE during the last 5 y of the project on the classroom 
teachers' knowledge, beliefs, and teaching practices related 
to teaching the geosciences. 

RESEARCH METHOD 

A mixed methods research approach was used to 
investigate the effect of the institute on the teachers. Using 


Onwuegbuzie and Leech's (2006) system for presenting 
mixed methods research questions, each question in this 
study has both a quantitative (QUAN) and qualitative 
(QUAL) component. The dominant research methodology 
used is capitalized, while the other method is shown in 
lowercase. Three mixed methods research questions were 
investigated: 

(1) What is the relationship between participation in 
the GIRE and teachers' knowledge of geoscience 
content (QUAN/qual)? 

(2) What is the relationship between participation in 
the GIRE and teachers' beliefs about teaching the 
geosciences (QUAN/qual)? 

(3) How are teaching practices impacted by participation 
in the GIRE (QUAL/quan)? 

To begin the study, the evaluator reviewed a wide 
variety of research and survey items related to the 
professional development of science teachers (e.g., Riggs 
and Enochs, 1990; Loucks-Horsley et al., 2009), teacher 
characteristics leading to student interest in the geosciences 
(e.g., Levine et al., 2002; Sherman-Morris et al., 2013), and 
established geoscience content knowledge assessments (e.g., 
Wagenet et al., 1999) when creating the pre- and post¬ 
institute survey instruments. All survey instruments were 
submitted to the University of Michigan-Dearborn Human 
Subjects Committee and received Institutional Review Board 
approval. The initial survey instrument was given to the 
teachers at the beginning of the GIRE and consisted of items 
from this literature base as well as additional items that were 
created to focus on specific goals of the study. These 
additional items gathered information about the educational 
background of the teachers, their preferred teaching style, 
their perceived geoscience knowledge, and their familiarity 
with different science careers. See an online supplement 
(available in the online journal and at http://dx.doi.org/10. 
5408/14-064sl) for the initial teacher survey. 

Several questions were included on the survey that dealt 
with geoscience concepts that were addressed during the 
GIRE, since one of the goals of the study was to improve 
teacher knowledge of geology, and specifically watershed 
issues. The survey also included 24 items from Wagenet et al. 
(1999) relating to watersheds. These questions focused on 
three of the six levels of Bloom's taxonomy of the cognitive 
domain: knowledge, application, and evaluation (Bloom et 
al., 1956). The nine knowledge questions required the 
teachers to recall discrete facts about watersheds, while the 
ten application questions assessed their ability to link 
different watershed features to watershed protection. Par¬ 
ticipants rated statements on these two levels as either 
correct or incorrect. The five evaluation level statements 
were measured with five-point Likert scale items (strongly 
agree, agree, neither agree nor disagree, disagree, strongly 
disagree) because there are elements of value or judgement 
for this cognitive level (Bloom et al., 1956). Issues relating to 
the validity and reliability of the watershed questions are 
described in Wagenet et al. (1999). Content validity was 
determined by having two GIRE geologists review each item. 
Any suggestions they made to improve an item were 
incorporated into the survey before it was administered to 
the teachers. 
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One of the goals of the research was to determine if the 
GIRE had an impact on teaching practices. Because it was 
beyond the scope of the grant project to measure the 
teachers' instruction in the classroom, an instrument was 
selected to measure the teachers' belief in their likelihood to 
teach the geosciences. The belief in one's ability to engage in 
a particular behavior is called self-efficacy and was first 
developed by Bandura (1977) as part of his social cognitive 
theory. 

The Science Teaching Efficacy Behavior Protocol (STEBI; 
Riggs and Enochs, 1990) was determined to be a suitable 
measure of self-efficacy and was included at the end of the 
survey. The STEBI is based upon Bandura's (1977) social 
cognitive theory, which states that self-efficacy beliefs 
motivate people towards specific actions and can be used 
to predict future behavior. A limitation is that the STEBI 
measures general science self-efficacy rather than geoscience 
self-efficacy, although it has been used in other studies 
relating to geoscience teaching self-efficacy (e.g., D'Alessio, 
2012; Posnanski, 2007). The STEBI includes 25 Likert-scaled 
statements relating to personal beliefs about teaching 
science, and it contains two subscales, personal science 
teaching self-efficacy beliefs (PSTEB) and science teaching 
outcome expectancy (STOES). The first subscale measures 
teachers' beliefs in their ability to teach science, while the 
outcome expectancy measures their beliefs that their 
teaching will make a difference in their students' under¬ 
standing of science. Riggs and Enochs (1990) provided a 
complete description of the reliability and validity measures 
for STEBI. 

A final measure of program impact and the primary 
source of qualitative data was the focus groups conducted on 
the last day of each summer's workshop. The focus groups for 
the teachers and the students were conducted separately by 
the project evaluator and other School of Education faculty 
experienced in this method of data collection. Over the length 
of the project, each focus group session lasted for an average 
of 45 min. Focus group methodology was appropriate in this 
evaluation research because it facilitated the exploration of 
diverse program impact by creating a collective conversation 
among participants. Focus groups also revealed the rich 
experiences of the participants in their own words and 
provided a depth of understanding about the GIRE experience 
that could not be determined using quantitative methods. The 
semistructured focus groups allowed for the exploration of 
issues such as content, program structure, personal meaning, 
social implications, and recommendations for improvement 
(Kamberelis and Dimitriadis, 2005). Sample questions includ¬ 
ed asking the teachers to describe what new geoscience 
content was learned and how they anticipated that the 
knowledge gained about their local environment would 
impact their professional and personal behaviors. The focus 
group questions remained constant each year, so that the 
discussion group responses could be compared across groups 
(Stewart et al., 2007). 

The quantitative measures were administered on the 
first and last days of the GIRE in order to measure change. 
The qualitative measure, the focus group, was held on the 
last day of the program. 

Participants 

A diverse set of teachers was recruited primarily through 
two channels: Information was sent to local urban school 


districts with high percentages of minority students and was 
posted on the Michigan Earth Sciences Teachers Association 
Web site. The selection process was conducted by the project 
director. First priority was given to teachers who were 
responsible for teaching Earth Science and did not have any 
college education in the geosciences. Second priority was for 
teachers who taught Earth Science or environmental science 
and had minimal geosciences coursework in college. There 
was also an attempt to balance the number of middle school 
and high school teachers selected as well as male and female 
participants. An average of 40 teachers applied each year. 
Twelve teachers were selected to attend each institute, and 
while each teacher had the option of bringing one or two of 
their students to the GIRE, the total number of institute 
participants (teachers plus students) could not exceed 18 due 
to the space limitations of the field research projects. This 
limited the teachers' ability to bring students to the institute. 
As the project progressed, fewer middle school science 
teachers and students were able to be recruited because the 
university incentive to offer college credit to students was 
extended solely to high school students. Conceptually, it also 
made sense to focus on the high school level because that is 
often when students are able to make the decision to enroll 
in science elective courses. Despite these constraints, by the 
end of the fifth year of the GIRE, 62 teachers and 36 students 
had participated in the project. Demographic characteristics 
are presented in Table I for the participating teachers and in 
Table II for the students. 


RESULTS 

The quantitative data were analyzed using the Statistical 
Package for Social Science (SPSS), version 22, to compute 
the difference between the scores on the pre- and 
postassessments. Before the tests were conducted, a 
preliminary analysis was conducted to test the assumption 
for normality. This was checked using both statistical 
(Shapiro-Wilk) and visual methods (Q-Q plots). In the 
cases where the assumption was not met, a nonparametric 
test was used, since both the preliminary analysis and the 
relatively small sample size indicated that this type of 
statistical text was most appropriate (Wells and Hintze, 
2007). 

The qualitative data, the focus group discussions, were 
analyzed using a content analysis approach and NVivo 
software. The focus group data were analyzed by transcrib¬ 
ing the discussions, reviewing the transcripts for accuracy, 
and manually coding the content into themes. Six themes 
were identified: knowledge of local community, change in 
teaching, change in personal behavior, impact on partici¬ 
pating students, geoscience content (watershed, groundwa¬ 
ter, brownfields), and suggestions for improvement of the 
GIRE. After this process of coding was completed by the 
program evaluator, a project staff member read over the 
transcripts and code labels and coded the text. This was done 
to establish the reliability of the coding system and is 
commonly used when analyzing focus group data (Stewart 
et al., 2007). There were only three coding differences (out of 
a total of 423 sections of coded text) between the two 
researchers. The final codes for these discrepancies were 
resolved by consensus after a discussion. See Fig. 1 for a 
graph of the total number of references for each of these 
themes. Totals for coding values exceed 100% because 
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TABLE I: Demographics of participating teachers. 


Demographic 

No. 

Teachers 

Gender 


Male 

15 

Female 

47 

Race 


African American 

16 

White 

46 

Educational Level 


Bachelor's degree 

17 

Master's degree 

43 

Doctorate 

2 

Number of College Level Geoscience Courses 


None 

6 

One 

20 

Two 

16 

Three 

7 

More than four 

13 

Classroom Teaching Experience 


1-5 years 

11 

6-10 years 

20 

11-20 years 

27 

21+ years 

4 

Grade Level 


Elementary 

4 

Middle school 

34 

High school 

24 

% Student Population Members of 
Underrepresented Minority Groups 


Less than 50% 

18 

50% 

18 

More than 50% 

26 


comments may have been coded for more than one theme. 
To prepare for reporting the findings of this analysis, 
exemplars for each theme were identified from the data 
(Hsieh and Shannon, 2005) and linked to one of the three 
research questions. 

The type of statistical tests performed and the results 
from the quantitative and qualitative analysis relating to each 
research question were compared, and areas of similarity as 
well as divergence were noted and are described next. 

Research Question #1: What Is the Relationship 
Between Participation in the GIRE and Teacher 
Knowledge of Geoscience Content? 

The teachers self-reported their general knowledge of 
the geosciences, watersheds, and brownfield sites using a 
five-point Likert scale (1 = no knowledge, 5 = excellent 
knowledge). Since the results of these three items were not 
normally distributed, a Wilcoxon matched-pairs signed-rank 
test was conducted to determine where there was a 


TABLE II: Demographics of participating students. 


Demographic 

No. 

Students 

Gender 


Male 

20 

Female 

16 

Race 


African American 

13 

White 

23 

Highest Grade Completed 


7th 

3 

8th 

11 

9th 

4 

10th 

5 

11th 

5 

12th 

8 

Intend to Attend College 


Yes 

35 

No 

1 

At Least One Parent College Graduate 


Yes 

19 

No 

17 


difference in the pre- and postreports of geosciences, 
watershed, and brownfield sites knowledge. Results of that 
analysis indicated that there was a statistically significant 
difference in the teachers' perceived knowledge in each of 
these areas (Table III). 

A McNemar test (Adedokun and Burgess, 2012) was 
used to determine if there was a difference in the pre- and 
postinstitute scores for the watershed issue questions at the 
knowledge and application levels, and it revealed no 
statistically significant difference in the scores for these 19 
questions. A test for normality indicated that the evaluation 
level questions were not normally distributed, so the 
Wilcoxon signed-rank test was used. There was no 
difference in the pre- and postinstitute scores at the 
evaluation level, with the exception of question 24, which 
asked the teachers to rank the level to which they agreed 
with the statement, "The quality of my drinking water 
doesn't have anything to do with drinking water quality in 
another town" (Table III). Therefore, the analysis illustrates 
that after the GIRE, the teachers were more likely to 
recognize that the quality of drinking water in different 
towns is related. 

A content analysis of the transcripts from the focus 
groups revealed four themes that were directly related to 
specific geoscience content knowledge (watershed, ground- 
water, brownfields, deep wells). Although the number of 
references to these specific topics was relatively low (Fig. 2), 
the comments made by the teachers indicated that they had 
gained knowledge of these geoscience concepts, which 
supports the results of the quantitative analysis. For instance, 
the teachers mentioned frequently that while they may have 
had a basic understanding of the concept of a watershed, 
they did not know how important a watershed was. One 
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TABLE III: Pre/postinstitute perceived knowledge. 


Survey Item 

Pretest 

Posttest 

Z Score 1 

Knowledge of geosciences 

3.0 (.78) 

3.8 (.61) 

—5.6*** 

Knowledge of watersheds 

2.7 (.83) 

4.1 (.61) 

-6.4“* 

Knowledge of brownfield sites 

1.9 (.81) 

3.9 (.71) 

-6.6*** 

"The quality of my drinking 
water doesn't have anything 
to do with drinking water 
quality in another town." 

4.6 (.55) 

4.9 (.28) 

-2.8** 


^ean (standard deviation) difference between pre- and post-GIRE Z score: 
** = P < 0.01, *** = P < 0.001. 


teacher expressed it this way, "I didn't realize how important 
the watershed is and what the area was until I came to this 
Geoscience Institute." Similarly, the teachers often com¬ 
mented that they had known about groundwater, but they 
did not think about it often, especially in terms of how easily 
it is polluted. 

The teachers mentioned that they had learned about the 
concept of brownfield sites in the context of their commu¬ 
nity, noting, for example, "all the different businesses that 
were located on these brownfield sites and how they have 
impacted the watershed, the Rouge River, and how 
businesses today are even impacting it in terms of polluting 
it or keeping it clean." 

While the teachers said their understanding of deep 
wells had increased, several mentioned that it would have 
furthered their understanding even more, as one teacher 
stated, to "see test wells, or using the purging system to 
clean up the site." 

Research Question #2: What Is the Relationship 
Between Participation in the GIRE and Belief in Ability 
to Teach the Geosciences? 

The relationship between participation in the institute 
and the teachers' beliefs in their ability to teach the 
geosciences was measured in part through a quantitative 
analysis of the STEBI scores. Neither the overall STEBI scores 


nor the two subscores were normally distributed, so again a 
nonparametric test was used to determine if there was a 
difference between the pre- and postinstitute scores. A 
Wilcoxon matched-pairs signed-rank test indicated that 
there was a difference in both the overall STEBI score and 
the PSTEB subscore. The overall STEBI as well as the PSTEB 
subscore decreased from pre- to postinstitute assessment 
(Table IV). There was no difference in the pre- and 
postinstitute STOES scores. While it may be surprising that 
the teachers' overall self-efficacy towards teaching science 
and their personal science teaching self-efficacy beliefs in 
general science teaching decreased, self-efficacy has been 
repeatedly demonstrated to be difficult to change. This is 
especially true for experienced teachers who have an initial 
STEBI score above 50 (Roberts et al., 2001). The institute 
teachers entered the program with an average STEBI score of 
52.1. In fact, Roberts et al. (2001) determined that a 4 week, 
activity-based professional development experience resulted 
in the largest gain in science teaching self-efficacy, especially 
for teachers who came into the professional development 
experience with low self-efficacy (<50 on their overall STEBI 
score) towards teaching science. One explanation for the 
decrease in scores can be found in the focus group 
transcripts. While the theme of changes in teaching the 
geosciences was most prevalent, a subtheme within it was 
barriers to teaching. While the institute experiences had an 
overall positive impact on the teachers, it may have also 
made them more aware of the barriers that would make it 
challenging to take their students out in the field. Frequently 
mentioned barriers were limited time for field trips or hands- 
on activities and lack of resources to transport the students 
to sites in the community. 

Two questions on the survey directly asked the teachers 
about their ability to teach the geosciences and their 
confidence in their ability to implement geoscience concepts 
in their classroom. For the first question, the teachers rated 
their ability to teach the geoscience using a five-point Likert 
scale in which 1 = no confidence, and 5 = excellent 
confidence. The second question asked teachers to rate how 


TEACHING CHANGE 
LOCAL COMM UNITY 
PERSONAL BEHAVIOR CHANGE 
IMPACT ON STUDENTS 
g BARRIERS (IN TCHING CHANGE) 
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0 20 40 60 80 100 
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120 


FIGURE 2: Frequency with which each theme was mentioned in focus groups. 
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TABLE IV: Pre/postinstitute perceived beliefs. 


Item 

Pretest 

Posttest 

Z Score 1 

Overall STEBI score 

52.1 (11.6) 

50.5 (9.1) 

-3.0** 

STOES subscore 

30.0 (7.3) 

29.6 (6.1) 

—1.4 ns 

PSTEB subscore 

22.1 (5.7) 

20.9 (4.4) 

_ „ *** 
-3.1 

Ability to teach 
geoscience concepts 

3.0 (.8) 

3.9 (.7) 

-5.5*** 

"I currently feel confident 
that I can incorporate 
geoscience concepts into 
my classroom." 

3.0 (.8) 

3.6 (.5) 

A /■*** 

-4.6 


*Mean (standard deviation) difference between pre- and post-GIRE Z score: 
ns = P> 0.05, ** = P < 0.01, *** = P < 0.001). 


well they agreed with the statement, "I currently feel 
confident I can implement geoscience concepts in my 
classroom setting," using a four-point Likert scale (1 = 
strongly disagree, 4 = strongly agree). While the two 
questions may appear similar, they were worded to assess 
different constructs. The first question is more abstract and 
general, whereas the second question asks teachers to rate 
their confidence in their ability to implement geoscience 
concepts in their classroom setting. That is, the question takes 
into account the dynamics of their present classroom 
environment. The results of a Wilcoxon signed-rank test 
showed the GIRE increased the teachers' confidence in their 
ability to teach geoscience in general and specifically in their 
classroom (Table IV). Although the GIRE had a positive 
effect on the teachers' belief in their ability to teach 
geoscience concepts, this contrasts with the lower post- 
GIRE science self-efficacy scores. These two disparate results 
raise the question of what other or additional types of 
professional development activities could increase the 
teachers' self-efficacy towards teaching science. 

The content analysis of the focus group transcripts 
revealed a subtheme within the "teaching changes" theme 
that provided support to the quantitative analysis that the 
teachers' confidence in their ability to teach geoscience 
concepts in their classroom had increased. A typical 
response within this subtheme was that the institute had 
provided them with specific content knowledge and local 
contexts in which to teach the geosciences. The teachers 
noted that they were able to connect their new knowledge 
and awareness of local issues to a passion for teaching the 
geosciences. One teacher expressed it this way: "We have a 
better background and a better understanding of things so 
we can lay the groundwork." 

Research Question #3: How Are Teaching Practices 
Impacted by Participation in the GIRE? 

The primary means of answering this research question 
was through the content analysis of the transcripts from the 
teachers' focus groups. Teachers were asked to self-report 
how the institute experience would impact their teaching 
practices, which was an intentionally broad construct and is 
historically viewed as a weaker measure of teacher behavior 
(Desimone, 2009). The analysis of focus group transcripts 
revealed a related theme of teaching changes. This theme 
was mentioned most frequently (Fig. 2) and occupied the 
largest percentage of the focus group time. In fact, nearly 
23% of the focus group discussion was related to how 


teachers anticipated their teaching would change as a result 
of participating in the GIRE. Three subthemes emerged 
related to what, how, and why the teachers would teach the 
geosciences to their students. 

What Geoscience Would Be Taught 

Numerous teachers spoke of feeling more comfortable 
by the end of the institute with the geoscience concepts and 
an ability to make them more relevant to the students 
because they had learned about specific community- 
environmental issues. One teacher noted, "I was trying to 
think what can I do to make geology important [to the 
students]. Now I can remind them that geoscience is a 
connection to everything." 

How Geoscience Would Be Taught 

Along with relating the geosciences to local issues, the 
teachers also described the importance of getting the 
students out in the field for hands-on learning. As one 
teacher reflected, "The fieldtrips were so important. They 
were such an eye-opening experience for us as teachers. It 
would have the same impact and even more on our 
students." 

Often teachers mentioned that there were resources 
right outside their classroom they could use to make the 
geosciences real. "Our school's right on the Clinton River 
watershed and it's literally a tenth of a mile away from our 
school. And now I'm going to have that time to do exactly 
what we did in the river. I'm just, really excited to be able to 
go into our backyard again and to incorporate all this 
knowledge." 

Although a number of teachers described within the 
subtheme of barriers why they were not able to take students 
outside, they did mention that the activity modules they 
participated in at the GIRE would be used in their 
classrooms. The teachers repeatedly mentioned the practi¬ 
cality of the modules, the easy obtainability of the needed 
supplies, and the benefit of having already done the module 
activities before attempting them with their students. 

Several teachers mentioned that after participating in 
the GIRE research projects, they now knew how to test for 
pollutants in their environment. One teacher mentioned 
these new research skills in terms of how she would teach 
her urban students, "I teach in Detroit where lead poisoning 
is significant. I will be able to tell them if they have 
contaminated soil at the home now. I can now not just teach 
science for the sake of these content standards but for real 
issues in their lives." 

The teachers stated that the field experience demon¬ 
strated the importance of understanding basic geoscience in 
order to make environmentally responsible decisions. This 
relates directly to the next theme—how the GIRE impacted 
the teachers' reasons for teaching the geosciences. 

Why Geoscience Would Be Taught 

Teachers reported that as they learned about the 
interconnectedness of the different parts of the local 
watershed, they realized that their individual actions do 
affect local water quality. This motivated the teachers to 
teach their students what they had learned at the GIRE 
because the information had relevance for more than 
standardized test scores. It was important for the students 
to learn on a very personal level. One teacher described 
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TABLE V: Pre/postinstitute perceived teaching practices. 


Item 

Pretest 

Posttest 

Z Score 1 

Ability to use inquiry-based 
instruction in the classroom 

3.6 (.8) 

4.0 (.8) 

3.8*** 

Using hands-on science 
modules is the best way 
for students to learn science. 

3.7 (.5) 

3.7 (.5) 

-0.0 ns 

It is important for students 
to have fieldwork experience. 

3.7 (.5) 

3.8 (.5) 

—1.4 ns 


^ean (standard deviation) difference between pre- and post-GIRE Z score: 
ns = P > 0.05, ** = P < 0.01, *** = P < 0.001). 


powerfully how she realized the importance of learning 
about local geoscience-related issues and how her thinking 
had changed, "seeing first hand, how a certain contaminant 
is transferring through groundwater or surface water, it's a 
scary thing to think about. This is going to affect us." 

There were three questions on the pre- and postsurveys 
that also focused on teaching practices. These questions 
were intended to assess how closely the participating 
teachers' confidence and future pedagogical techniques 
mirrored the instructional techniques modeled in the GIRE. 
The first question asked the teachers to rate their confidence 
in their ability to use inquiry-based instruction in the 
classroom, using a five-point Likert scale (1 = no confidence, 
5 = excellent confidence). The other two questions asked 
teachers to rate their level of agreement, on a four-point 
Likert scale (1 = strongly disagree, 4 = strongly agree), with 
these statements: (1) Using hands-on science modules is the 
best way for students to learn science. (2) It is important for 
students to have fieldwork experience. None of the data for 
these three questions met the assumption of normality, so a 
Wilcoxon signed-rank test was used to analyze the data. 
Results indicated that the only change in the teachers' pre- 
and postinstitute responses was an increase in their 
confidence in their ability to use inquiry instruction in the 
classroom (Table V). 

This increase in confidence could be a reflection of the 
inquiry activities that both the teachers and students 
engaged in at the GIRE. Because there were middle and 
high school students participating in the activities alongside 
the teachers, the teachers were able to view the activities 
from the perspective of both the instructor and the student. 
The lack of statistically significant differences in the other 
two areas of teaching practices could be due to limitations 
within the institute, or it could be a reflection of the ceiling 
effect, as the teachers had agreed with the value of hands-on 
science modules and that it was important for students to 
have fieldwork experience even before they participated in 
the GIRE. 


DISCUSSION 

Overall, both the quantitative and qualitative data 
indicate that there is a positive relationship between the 
GIRE and increases in the teachers' perception of their 
knowledge, beliefs in their ability to teach the geosciences, 
and anticipated changes in teaching practices, although the 
relationship was not always definitive. This is indicated by 
the disparate results between the quantitative and qualitative 
data and is not conclusive due to research limitations. 


Nearly all of the teachers expressed how they had 
gained new knowledge and a corresponding increase in their 
commitment to make the science curriculum relevant to 
their students as they learned about pollution occurring on 
specific research sites in their local community. Teachers 
mentioned that after the institute, they had the knowledge 
and skills to use place-based pedagogy by making links to 
similar sites in their students' local environment. In contrast, 
the quantitative results were mixed; the teachers' self-report 
of increases in geoscience knowledge contradicted the 
results from the watershed survey, which did not indicate 
a change in knowledge. The lack of change in knowledge 
could be a result of the geoscience and watershed 
knowledge survey items being instructionally insensitive to 
the GIRE activities even though the assessment items were 
validated by participating geoscience faculty. Instructional 
sensitivity can be loosely defined as the degree to which 
scores on an assessment reflect the quality of the instruction 
(Popham, 2013). This represents a limitation of the research. 
In the future it would be useful to develop quantitative 
assessments based upon the professional development 
activities. The findings from the qualitative measures could 
provide guidance in developing specific items to measure 
knowledge and beliefs. 

The qualitative data and the results of a quantitative self- 
report instrument indicated that the teachers did increase 
their belief in their ability to teach geoscience content in 
general, as well as to the students in their classroom. In 
contrast, results using an instrument designed to measure 
teacher self-efficacy towards teaching general science (the 
STEBI) and the teachers' personal science teaching self- 
efficacy beliefs (PSTEB) decreased. The PSTEB subscale 
reflects teachers' confidence in their ability to teach science. 
There was no change in the STOES subscale score, which 
reflects teachers' beliefs that student learning can be 
influenced by their effective teaching. It would be interesting 
to investigate further why the participating teachers believed 
less in their ability to teach science after the institute than 
they had before. Perhaps the teachers overestimated their 
ability to teach science at the beginning of the institute and 
once exposed to the large quantity of geoscience information 
presented at the GIRE, they realized how much more there 
was to know. Alternatively, the measure itself may need to 
be reevaluated given the current emphasis on inquiry-based 
science teaching (Wheatley, 2005). Other studies have 
reported similar decreases in STEBI scores after professional 
development activities (e.g., Holden et al., 2011). It has also 
been argued that self-efficacy is context-specific (Bandura, 
1977), and the STEBI is not focused solely on teaching the 
geosciences. A third explanation for the decrease in scores 
relates to a limitation of the study—that the STEBI does not 
target beliefs towards teaching the geosciences specifically. 

Teachers reported qualitatively that their teaching 
practices were positively impacted by the institute in terms 
of what aspect of the geosciences would be taught, how they 
would teach it, and why they believed it was important for 
their students to understand. The quantitative data revealed 
that the teachers came into the institute possessing 
confidence to use inquiry-based teaching approaches and 
positive beliefs that students should learn science through 
hands-on and field experiences. Not surprisingly, given the 
initial high scores on these survey items, the postinstitute 
scores were not higher than the pre-institute scores. The 
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reliance on teacher self-report of future classroom behavior 
represents a limitation of the study. Although self-reporting 
is often used in evaluation research, the accuracy of self- 
reports data is open to debate, especially if it relates to pro- 
environmental behaviors such as water conservation or 
energy usage (Corral-Verdugo, 1997; Steg and Vlek, 2009). 
Classroom observation of teaching behavior and the 
geoscience content being taught would be a more direct 
measure of program impact, although, like self-reported 
behavior, it is difficult to measure objectively (Desimone, 
2009). The study would be further strengthened if the 
teachers' geoscience teaching behaviors were observed 
before and after the GIRE and compared to the teacher's 
self-reported behavior. 

While the results of the mixed methods research 
approach do not point directly to one clear conclusion 
concerning the relationship between teacher participation in 
the institute and positive changes in their knowledge, 
beliefs, and teaching practices related to teaching the 
geosciences, they do illuminate and reflect the complex 
impacts that a geoscience professional development activity 
that focuses on environmental research in the local 
community can have upon science teachers. Both the 
quantitative and qualitative data provide important infor¬ 
mation about the effect of the institute upon the teachers. 
For instance, the quantitative data provide an indicator of 
teacher knowledge of specific geoscience content, while the 
qualitative data provide a more personal perspective of the 
impact of the GIRE upon the teachers in their own words. 
The profound effect of the institute was evident during the 
focus group sessions, in which several teachers described the 
GIRE experience as "life-changing." 

IMPLICATIONS AND RECOMMENDATIONS 
FOR FURTHER RESEARCH 

The mixed methods approach used in the study 
represents an initial attempt at understanding the relation¬ 
ship between placed-based geoscience professional devel¬ 
opment activities and teacher content knowledge, beliefs, 
and teaching practices. Mixed methods research is an 
appropriate first step in evaluating the effectiveness of the 
GIRE, as the inconclusive results indicate that further 
research is needed to more rigorously understand program 
impact upon the teachers. In particular, instruments need 
to be developed or selected based on their instructional 
sensitivity and linkage to the professional development 
activities. The themes identified in the focus group 
discussion can be used to develop quantitative measures 
to assess beliefs about teaching the geosciences or about 
teaching practices. If the ultimate goal of the professional 
development is to improve student learning and interest in 
the geosciences, particularly for underrepresented groups, 
then a conceptual framework such as the one advocated by 
Desimone (2009) would be useful to implement in later 
research. The model is based upon an extensive review of 
impact studies of teachers' professional development. It 
allows for the investigation of the relationship between the 
proven features of professional development (content 
focus, active learning, coherence, duration, and collective 
participation), increased teacher knowledge and skills, 
change in attitudes and beliefs, changes in instruction, 
and changes that foster increased student learning (Desi¬ 


mone, 2009). Use of this model would provide a common 
framework that would be helpful when designing individ¬ 
ual studies that research the impact of teacher professional 
development programs, and it would also facilitate the 
comparison of research studies by using a common 
language. 

Despite being a preliminary study, the results of this 
research can be useful to geoscience educators designing 
professional development programs for teachers and for 
those interested in environmental education because they 
indicate that the GIRE motivated teachers to become 
environmental stewards, even when that was not the 
predominant goal of the professional development experi¬ 
ence. That result again weaves back to the importance of 
placing the geoscience content in a local context in which 
the teachers and students have a personal connection, 
which is important for all program developers to be mindful 
of as they create geoscience education programs. 
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